eng-557 build centralized linting system for all repos #10

CDeltakai · 2025-03-28T03:23:40Z

Description

This PR will implement a centralized linting system which will allow all MatrixAI repos to import from a single linting package (this one) and apply appropriate linting rules without further intervention. It should also allow downstream repos to extend the system in order to ignore certain rules or add project-specific rules.

Issues Fixed

Relates Create centralized linting system that can be applied to all MatrixAI repos, regardless of project-specific linting rules #7

Tasks

🔧 Repository Refactor

1. Rename the repository from js-eslint to js-lint
2. Update package.json:
- 2.1 Change package name to @matrixai/lint
- 2.2 Add a bin entry:
```
"bin": {
  "lint": "dist/bin/lint.js"
}
```

🧰 Lint Command Implementation

3. Create src/bin/lint.ts
4. Follow structure and logic of zeta.house/scripts/lint.mjs
5. Implement lint runners for:
- 5.1 eslint for JS/TS files
- 5.2 prettier for markdown
- 5.3 shellcheck for shell scripts
- ~~[ ] 5.4 clang-format for C/C++~~
- ~~[ ] 5.5 nixfmt for Nix files~~
- ~~[ ] 5.6 rustfmt for Rust files~~
6. Ensure the script:
- 6.1 Checks if each native dependency exists before executing
- 6.2 Uses a fast, cross-platform commandExists utility

🧪 Add Utility Function

7. Implement commandExists(programName: string): boolean that:
- 7.1 Works on Linux, macOS, and Windows
- 7.2 Is fast and lightweight
- 7.3 Returns true/false for command existence
7A. We need this package to be extendable by downstream repos as well so they can extend the package and ignore certain rules or implement custom rules if need be.

📦 Package Dependencies

8. Add all relevant eslint, @typescript-eslint, prettier, and related dependencies
9. Refer to zeta.house package.json as a baseline
10. Do not include native tools like shellcheck or clang-format in package.json

🔁 Plugin De-emphasis

11. Do not export an ESLint plugin as the main export
12. Optionally retain plugin code for internal use only

🚀 Prepare for Release

13. Tag the repo with version v0.0.1 (CI will handle publishing)

🌱 Downstream Integration

14. Create a feature branch in js-logger
15. Integrate @matrixai/lint and verify it runs correctly
16. Optionally filter directories or extend the lint command if needed

Final checklist

…onfig

…t config file for new linter version

…tra export for matrixaiPlugin in index.ts

…rserOptions from config and creating new typeaware specific recommended configs

[ci skip]

… it into the normal recommended ruleset

linear · 2025-03-28T03:23:42Z

ENG-557

… run eslint with a config override based inside the package when the lint script is run by a downstream repo when it is imported

…e they dont work and will need to be implemented later

…use their include folder to determine what patterns of folders to lint

src/runners/runESlint.ts

src/utils/resolveMatrixConfig.ts

eslint.config.mjs

package.json

src/utils/resolveMatrixConfig.ts

CMCDragonkai · 2025-04-14T01:09:49Z

Let's get this merged after fixing those minor things, and then integrate into js-logger as a start.

CMCDragonkai · 2025-04-14T03:52:49Z

Request @tegefaulkes to review once you are finished cleaning up this codebase in relation to conventions. Make sure to review js-logger as a template for how you should structure things, finish up the above reviews accordingly.

…til.ts files

eslint.config.mjs

tegefaulkes · 2025-04-14T08:57:38Z

src/bin/lint.ts

This is basically functioning as a CLI program wrapping all our needs right?

We should consider using Commander to handle the arg parsing and logic. That said its a bit much to address right now so we can make a new issue for tracking it.

src/bin/lint.ts

tegefaulkes · 2025-04-14T09:01:23Z

src/bin/lint.ts

+      console.error(
+        `--config points to “${explicitConfigPath}”, but that file does not exist.`,
+      );
+      process.exit(1); // Hard‑fail; nothing to lint against
+    }


I'm thinking we need to use a similar error structure to Polykey. This works but I'd rather have an explicit error that gets thrown and the message/exit code is handled automatically. That would be part of a Commander conversion if we do it.

src/utils/utils.ts

tegefaulkes · 2025-04-14T10:09:17Z

src/utils/utils.ts

+
+  const cfg = rawCfg as { tsconfigPaths?: unknown; forceInclude?: unknown };
+


Don't do this, You're throwing type checking out of the window. It's also kinda pointless? You're saying an arbitray JSON object May have these keys and they are unknown? That's already true.

This is is a json file with an expected format then we need proper validation. Polykey already uses https://www.npmjs.com/package/ajv to apply schema to it's status file. We can use the same thing here.

This may be another add on issue.

Don't do this, You're throwing type checking out of the window. It's also kinda pointless? You're saying an arbitray JSON object May have these keys and they are unknown? That's already true.

I've changed this to use a partial type with RawMatrixCfg:

type RawMatrixCfg = Partial<{ tsconfigPaths: unknown; forceInclude: unknown; }>;

and then casting it to this type with the asRawMatrixCfg method:

function asRawMatrixCfg(v: unknown): RawMatrixCfg | undefined { return typeof v === 'object' && v !== null ? (v as RawMatrixCfg) : undefined; }

as a basic type guard. I think this will fix this issue in the short term but I'll keep this open in case we want a proper issue with the proper validation fix.

src/utils/utils.ts

tests/index.test.ts

…self and removed the default eslint.config.mjs since its not needed anymore

…aving an invalid config json hard fail instead of falling back and removed unnecessary arrow functions

fixed potential race condition as well as eslint exiting the process prematurely if any error was found, preventing shellcheck or prettier from running

… and removed unnecessary interface. removed ignoring files console log

…orresponding file

… removed the main index.ts since nothing from there is being exported anymore. made a seperate plugin folder for the matrixai eslint plugin that contains our custom rules

…order to fix CI

CDeltakai and others added 19 commits March 21, 2025 12:54

feat: initial commit

f6418cb

fix: deleted old index.js file

6b934a1

fix: update @typescript-eslint dependencies to latest version

c8ecb57

feat: update ESLint plugin structure and integrate with recommended c…

626a974

…onfig

feat: updated all packages in repo including linter and created a fla…

a482d69

…t config file for new linter version

fix: ran lint f

cbe4f51

fix: removed bench and test from package as they are unneeded

bd41999

fix: changed the config bundle to be js instead of ts, removed the ex…

16c7e50

…tra export for matrixaiPlugin in index.ts

fix: fixed import ordering

d25a039

wip: replacing config bundle with config from zeta.house, removing pa…

ed540cf

…rserOptions from config and creating new typeaware specific recommended configs

fix: did a lintfix and removed unnecessary comments

ec8c017

fix: did a lintfix

d98df43

fix: fixed up formatting for eslint config

f385fa4

feat: adding generic test files and necessary packages

b39e15e

wip: fixing up ESM

5fd8914

[ci skip]

fix: fixed issue with jest test not working correctly

ae54301

refactor: removed the recommendedTypeaware ruleset and instead merged…

99a7bc6

… it into the normal recommended ruleset

fix: removed unused import

3073893

fix: deleted old .eslintrc file since it was no longer needed

bd7cb3b

CDeltakai self-assigned this Mar 28, 2025

CDeltakai changed the title ~~Feature eng 557 centralized linting system for all repos~~ eng-557 build centralized linting system for all repos Mar 28, 2025

CMCDragonkai mentioned this pull request Mar 28, 2025

Build independant linting system that will automatically lint all repos any repos in MatrixAI once imported #9

Closed

7 tasks

CDeltakai added 7 commits April 1, 2025 14:30

feat: implementing lint script and an external eslint runner that can…

3d2e8d1

… run eslint with a config override based inside the package when the lint script is run by a downstream repo when it is imported

wip: testing changes to fix issue with lint.ts not running correctly

0cd35e5

wip: managed to get command mostly working, ironing out the kinks

97234e9

wip: adjusting parameters to test for tsconfig related issues

552430b

wip: commenting out the shellcheck and prettier commands for now sinc…

a0892c0

…e they dont work and will need to be implemented later

wip: made runESlint detect tsconfig files inside the target repo and …

9a2570c

…use their include folder to determine what patterns of folders to lint

feat: implemented working shellcheck linting and prettier linting

a3b75e3